Skip to content

SCRUM-1160-adapters(bigquery): retry once on the exact bytes-billed shortfall - #331

Open
emnarahmouni-exmergo wants to merge 1 commit into
mainfrom
SCRUM-1160-issue-320
Open

SCRUM-1160-adapters(bigquery): retry once on the exact bytes-billed shortfall#331
emnarahmouni-exmergo wants to merge 1 commit into
mainfrom
SCRUM-1160-issue-320

Conversation

@emnarahmouni-exmergo

Copy link
Copy Markdown
Contributor

Closes : #320

explore query failed inside its own confirmed budget on
BigQuery, because the server-side maximum_bytes_billed cap was pinned to
this command's own reservation against the cumulative session_ceiling
(sized to the dry-run estimate), not the wider per-command budget the
operator actually confirmed.

Root cause

CostGate.remaining_for_statement() bounds the per-statement cap by
min(effective_ceiling(), self._reserved). That clamp to _reserved is
deliberate (issue #159): without it, two commands sharing one
session_ceiling could each claim the full remaining session headroom and
jointly overspend it. The reservation is sized to the dry-run estimate,
and charge() already widens it when a fresh dry run drifts higher.

The gap #320 hits is different: BigQuery's own execution-time rounding of
bytes billed can exceed any dry-run estimate, however accurate that
estimate was at dry-run time. No dry run predicts it, so the existing
drift-widening never catches it, and the cap stays pinned to the original
(too-tight) reservation even though the confirmed budget has ample room.

The fix

BigQuery's refusal already states the exact byte count it needed:
"Query exceeded limit for bytes billed: 163595928. 164626432 or higher required." On that refusal, BigQueryAdapter._run now parses the
required count, widens the charge by the gap between it and what was
already charged (going through the same locked charge()/admission path
an ordinary estimate drift already uses), and retries the statement once
with the new cap.

  • If the confirmed ceiling has room for the real requirement, the widening
    succeeds and the retry completes within budget.
  • If it doesn't, the widening itself raises OverCeilingError (the exact
    behavior today, just now attributing the refusal to the true byte count
    rather than a too-tight reservation), and no second server call is made.
  • The session_ceiling is not enforced under concurrency: the cost gate snapshots spend at construction #159 concurrency guarantee is untouched: the retry widens through the
    identical shared, locked reservation mechanism, so two commands sharing a
    session ceiling still can't jointly overspend it.
  • Capped at one retry, so a persistent failure can't loop.
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

explore query: maximum_bytes_billed is capped at the estimate, so a multi-table statement fails inside its own confirmed budget

1 participant